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Abstract 

Background: Some aspects of the neighborhood built environment may influence residents' physical activity, 
which in turn, affects their health. This study aimed to develop an urban built environment evaluation tool and 
conduct necessary reliability and validity tests. 

Methods: A 41 -item urban built environment scan tool was developed to objectively assess the neighborhood built 
environment features related to physical activity. Six neighborhoods in Hangzhou were selected from three types of 
administrative planning units. A pair of auditors independently assessed all of the 205 street segments at the same 
time. Half of the segments (n = 1 04) were audited twice by the same auditor after a two-week time interval. 
Inter-rater reliability was assessed by comparing the audits of paired observers, while intra-rater reliability was 
evaluated by comparing an auditor's repeated assessments of the same segments. The construct validity was tested 
using factor analysis. 

Results: The inter-rater reliability for most items was above 0.8. The intra-rater reliability for most items was above 
0.4, and was lower than corresponding inter-rater reliability. Six factors were extracted by factor analysis and the 
factor loading matrix showed good construct validity. 

Conclusions: The CUBEST is a reliable and valid instrument that can be used to assess the physical activity-related 
built environment in Hangzhou, and potentially other cities in China. 
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Background 

Physical inactivity is an important, modifiable behavioral 
risk factor for non-communicable chronic diseases [1]. 
Epidemiological studies have shown that physical in- 
activity is associated with increased risks of obesity, dia- 
betes, cardiovascular disease, and other chronic diseases 
[2,3]. A growing number of studies have focused on the 
ecological context of physical activity [4], i.e. the influ- 
ence of the residential built environment on physical ac- 
tivity patterns [5]. 
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The built environment — the physical form of commu- 
nities usually consists of 6 dimensions: (1) residential 
density; (2) street connectivity; (3) accessibility to desti- 
nations (land-use mixed) and services; (4) walking and 
cycling environment; (5) aesthetic quality; and (6) safety. 
Together, these elements shape access to opportunities 
for physical activity. Three types of data are usually used 
to measure built environment attributes believed to be 
related to physical activity: (1) perceived (subjective) 
measures obtained by personal interview with question- 
naire; (2) observational (objective) measures obtained 
using systematic scans or audits; and (3) archival data 
sets that are often layered and analyzed with GIS [6]. 

China is experiencing unprecedented rates of urbanization, 
which have contributed to the decline of physical activity [7] 
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and dramatically changes in built environment. For example, 
sprawling development, transportation infrastructures that 
emphasizes driving, and inconvenient, unsafe, and unpleas- 
ant public open spaces have been discouraging active trans- 
portation behaviors like walking and bicycling [8]. Thus, it is 
critical to accurately assess built environment characteristics 
before understanding how they correlate with physical activ- 
ity in China. Some instruments have been developed for 
assessing the built environment in developed countries, yet 
few have been done in China [9]. Furthermore, unique built 
environmental features are presented in Chinese and other 
Asian ultra-dense cities, which could bring about different 
associations with physical activities. These include but not 
limited to mixed types of residential housing; crowdedness; a 
complex public transport network; a great number of over- 
passes; bike parking facilities; man-made obstacles on side- 
walks and bike lanes; and unique types of destinations (e.g., 
historical buildings, tea houses, open-air food outlets). In 
additional, even the same feature of built environment could 
correlate with physical activity differently between in China 
and other Western countries. For example, positive relation- 
ship is usually observed between residential density and 
physical activity in the Western countries. But studies have 
suggested the possibility that densely settled Chinese cities 
could hinder leisure-time physical activity due to decreased 
availability of physical activity resources and increased con- 
cerns about traffic safety [10]. These facts show that the 
application of current instruments on assessing Chinese 
urban built environment is limited. Therefore, it is of vital 
importance to make necessary adjustments to have an in- 
strument specifically designed for urban built environment 
in China. 

In this study, we created an adapted instrument (ob- 
servational measures obtained using systematic scans or 
audits) based on a review of existing reliable auditing 
tools. This instrument is specifically designed for urban 
built environment related to adult physical activity in 
mainland China, that is, the China Urban Built Environ- 
ment Scan Tool (CUBEST). This article sought to report 
the reliability and validity assessment results of the 
CUBEST conducted in the city of Hangzhou, China. 

Methods 

Study site 

The city of Hangzhou, which is the capital of Zhejiang 
Province, is situated in the southeast coastal area of 
China. Hangzhou exercises jurisdiction over eight dis- 
tricts, three county-level cities, and two counties. It is an 
economically developed city in China, and its compre- 
hensive economic strength ranked eighth among all 
large- and medium-sized cities in China in 2011. By the 
end of 2011, the population of long-term residents in 
the city was 8.74 million, of which the urban population 
is 6.45 million, accounting for 73.9% [11]. Two districts 



located in a central geographic location of Hangzhou 
were included to test the reliability and validity of the 
CUBEST, i.e., Shangcheng District and Xiacheng District. 

All administrative planning units in these two districts 
are classified into five categories [12] based on the de- 
gree of land-use mix and service capacity of public 
buildings. Public buildings usually consist of buildings 
used for government, commercial, educational, transport 
and health care purposes. A Type I unit is characterized 
by fully developed commercial and residential areas with 
dense population and mixed land use. A Type II unit has 
developed but scattered public buildings as a feature, 
lacking of comprehensive service capacity. A Type III 
unit is featured by partly developed and single functional 
public buildings. Type IV and type V units are mainly 
composed of farmland and industrial storage warehouses 
and were excluded from this study. A typical neighbor- 
hood in most urban areas of China usually shows a 
shape of square or rectangle with 0.2 to 0.5 square kilo- 
meters in area. In this study, we extended 400 meters 
out from each side of the neighborhood boundaries to 
form a study area with 1.0 to 1.5 square kilometers in 
area. Two neighborhoods were selected in each of the 
three types of units and all the street segments in these 
6 extended study areas were evaluated using environ- 
mental audit instrument. A street segment was defined 
as a section of street or road between two intersections 
with a maximum length of 400 meters. 

Development of the CUBEST 

The CUBEST was designed based on a review of existing 
reliable instruments which have been developed since the 
year 2000, including Analytic Audit Tool [13], Active 
Neighborhood Checklist [14], SSO [15], PIN3 Neighbor- 
hood Audit Tool [16], Irvine - Minnesota Inventory [17], 
NALP [18], EAST_HK [9], SPACES [19], PEDS [20], 
WABSA [21], Sidewalk Assessment Tool [22], and PARA 
[23]. Table 1 shows the information on the above- 
mentioned instruments assessing neighborhood built en- 
vironment related to physical activity. An item pool was 
generated mainly from the Analytic Audit Tool, Active 
Neighborhood Checklist and EAST HK after consider- 
ation of the date of development, sample size, compre- 
hensiveness of dimensions, survey method and time cost 
of current instruments. Six dimensions were involved in 
the CUBEST, including residential density, street con- 
nectivity, accessibility (land-use mix), sidewalk quality, 
bike lane quality, and aesthetic. Items belonged to the 
dimension of safety from traffic were integrated into the 
dimensions of sidewalk and bike lane quality. Meanwhile, 
necessary adjustments were made to fit in the character- 
istics of Chinese urban settings. Items with Chinese and 
local characteristics were added, such as "Cycling or 
walking against the flow of traffic", "Cultural features 
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Table 1 Information on 12 instruments assessing neighborhood built environment related to physical activity 



Instrument 



Number 
of items 



Dimensions 



Analytic Audit Tool 27 

Active Neighborhood 89 
Checklist 

EAST_HK 91 

rvine - Minnesota Inventory 162 

Neighborhood Active 18 
Living Potential (NALP) 

Pedestrian Environment 36 
Data Scan Tool (PEDS) 

Physical Activity Resource 43 
Assessment Instrument (PARA) 

PIN3 Neighborhood Audit 43 
Instrument 

Sidewalk Assessment Tool 5 



Systematic Pedestrian and 51 

Cycling Environment Scan 

(SPACES) 

Systematic Social Observation 1 26 

(SSO) 

Walking/Bicycling Suitability 16 + 27 

Assessment Form (WABSA) 



Time required Residential Accessibility Street Sidewalks Bike Aesthetic Safety 
density connectivity lane quality 



10.6 min/ 
segment 

11.7 min/ 
segment 

12.38 min/ 
segment 

20 min/ 
segment 

No report 

3-5 m in/400 ft. 
segment 

10 min/ 
location 

No report 

8-12 min/ 
segment 

20 m in/Km 



5-10 min/ 
segment 

No report 



✓ 
✓ 
✓ 
✓ 

✓ 
✓ 



✓ 



✓ 
✓ 
✓ 
✓ 
✓ 

✓ 



✓ 



✓ 
✓ 
✓ 



✓ 
✓ 
✓ 
✓ 
✓ 



✓ ✓ 
✓ 

✓ ✓ 



(historical buildings) in the neighborhood", "Government 
sponsored public recreational equipments", "Tea house", 
"West Lake or Beijing-Hangzhou Grand Canal". Physical 
activity facilities were categorized to 9 kinds of destina- 
tions based on the PANES questionnaire [24]. 

The draft of the CUBEST contained 49 items under 6 
dimensions, including residential density (4 items), street 
connectivity (3 items), accessibility (21 items), sidewalk 
quality (6 items), bike lane quality (8 items), and aesthet- 
ics (7 items). 

On-site evaluation 

The validation of the CUBEST was conducted in Hangzhou 
City from October to December in 2011. In-the-field audits 
of street segments was conducted by two raters who were 
also involved in the development of the CUBEST. A stand- 
ard operating procedure for environmental audit was devel- 
oped using detailed written instructions and field pictures 
to achieve uniformity in the performance of evaluation. A 
two-day intensive rater training was developed with the 
goal of making them become proficient at completing the 
measures, including explanation of the principles, oper- 
ation, potential problems and solutions of the CUBEST and 
GPS positioning device. Before the evaluation was begun 
across the whole sample, two raters independendy finished 



a pilot evaluation of two neighborhoods (about 60-80 
street segments) using the CUBEST. Any discrepancies 
were resolved by another one of the developers of the 
CUBEST. In the formal survey, the two raters independ- 
ently performed evaluations of a total of 205 street 
segments at the same time. Approximately half of the 
street segments (n = 104) were randomly sampled and in- 
dependently scanned by the two raters for the second 
time after a two-week time interval. All environmental 
scans were conducted during daylight hours. The average 
time required for data collection was 7.4 minutes per 
segment. 

Data analysis 

Intra-rater reliability (i.e. test-retest reliability) was evalu- 
ated by the consistency of judgments made by the same 
rater over a period of time, and inter-rater rater reliability 
was assessed based on the level of agreement achieved by 
independent raters. The intra-class correlation coefficients 
(ICCs) were calculated to assess reliability for continuous 
and ordinal variables. Agreement among dichotomous var- 
iables was assessed using Cohen's Kappa statistic [25]. The 
adjectival ratings suggested by Landis and Koch [26] in the 
following categories were followed: 1.0 to 0.8 (almost per- 
fect agreement), 0.8 to 0.6 (substantial agreement), 0.6 to 
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0.4 (moderate agreement), 0.4 to 0.2 (fair agreement), and 
0.2 to 0.0 (poor agreement). The prevalence adjusted and 
bias adjusted Kappa (PABAK) coefficient was used to deal 
with items having high prevalence index or bias index [27]. 
For items with zero or little variance, the percentage of ob- 
served agreement of two raters was used to assess the reli- 
ability. The criterion for good level was set to no less than 
75%, as used by Pikora and colleagues [19]. 

Construct validity refers to the extent to which an in- 
strument measures what it claims to measure. Exploratory 
factor analysis (EFA) with equamax orthogonal rotation 
was applied to assess construct validity. Factor analysis ap- 
propriateness was assessed by Kaiser-Meyer-Olkin (KMO) 
value (greater than 0.7), and with a significant level less 
than 0.05 for Bartlett's test of sphericity. Factor loadings 
with an absolute value greater than 0.40 were consid- 
ered to be significant. The PASW version 18.0.0 (IBM 
Corporation, Somers, NY, USA) was used for data analysis. 

Results 

Inter-rater reliability 

A total of 309 street segments (205 for the first scan and 
104 for the second) were assessed to test inter-rater reli- 
ability. The results of inter-rater reliability are shown in 
Table 2 and Table 3. Seven items were not suitable for 
computing Kappa statistic or ICCs due to zero variance 
or data distribution characteristics; nevertheless, all of 
them had very high percentage of agreement (all > 85%). 
For items with adequate variance, the Kappa statistic 
was used to assess intra-rater reliability of 25 dichotom- 
ous items and the ICCs were used for 17 ordinal items. 
For Kappa values, eighteen out of 25 (72%) dichotomous 
items reached almost perfect level of agreement, and the 
rest 7 items (28%) also had a substantial level of agree- 
ment. Twelve out of 17 (71%) items using ICCs reached 
good level of agreement, and 3 items (22%) were in fair 
level. Two items, "Evenness and Bike lanes maintenance" 
and "Cigar butts or discarded cigarette", showed poor re- 
liability. All the six dimensions reached good level of 
inter-rater reliability (all ICCs > 0.75, p< 0.001). When 
the PABAK coefficient was applied, the inter-rater reli- 
ability showed varying degrees of improvement, espe- 
cially for the items with higher prevalence index. 

Intra-rater reliability (test-retest reliability) 

A total of 208 street segments (104 segments for each 
rater) were evaluated to test intra-rater reliability. Table 2 
and Table 4 indicate the results of intra-rater reliability 
for each item. The percentage of agreement was calcu- 
lated for 9 items with zero variance or specific data dis- 
tribution. And all of them had very high percentage of 
agreement (Evenness and sidewalks maintenance = 83%, 
rest 8 items > 90%). For the 24 dichotomous items with 
adequate variance, Kappa statistic was used to assess 



intra-rater reliability and ICCs were used for other 16 
ordinal items. Nineteen out of 24 (79%) items using 
Kappa statistic reached at least moderate level of intra- 
rater reliability. Four items showed fair level and one 
item fell into the poor category ("Abandoned building or 
vacant lot"). ICCs for the 6 dimensions were: residential 
density (0.563), street connectivity (0.721), accessibility 
(0.758), sidewalk quality (0.360), bike lane quality 
(0.587), and aesthetic (0.710), all of which were statisti- 
cally significant (all p < 0.001). After calculation of the 
PABAK coefficient, fifteen out of 23 items reached 
higher levels of intra-rater reliability. 

Construct validity 

The KMO value in the present analysis was 0.758, which 
was within the range of "acceptable" values. The Bar- 
tlett's test of sphericity was 1593 and significant at p < 
0.001, which indicated a highly significant correlation 
among the survey questions. This information allowed 
us to identify the factor model using the EFA approach. 

The EFA yielded 6 factors with eigenvalues greater 
than 1, which explained 60.0% of the variance. After 
suppressing small coefficients with absolute value below 
0.40, the rotated component matrix is shown in Table 5. 
Almost perfect matching was found between 6 compo- 
nents and the corresponding items designed for them, 
which suggests very good construct validity of the CUB- 
EST. But still, the main factor loadings of a few items 
(or summary scores of items) deviated from their ori- 
ginal design. These were "Presence of bus or other tran- 
sit stops" designed for street connectivity, "Evenness and 
sidewalks maintenance" designed for sidewalk quality, 
"Crossing aids for pedestrians and bicyclists" designed 
for bike lane quality, and "Cultural features" and "Nature 
sights" designed for aesthetic. 

Final Scan Tool 

Six dimensions and 41 items were included in the final 
CUBEST specific to Hangzhou City (Ql to Q41), as 
shown in Table 6. Items with zero or little variance were 
excluded because of little distinction among characteris- 
tics of street segments in Hangzhou, which includes 
"Abandoned building or vacant lot", "Slope of sidewalks", 
"Slope of bike lanes", "Whole or broken liquor bottles or 
cans", and "Unattended pets or lost animals". However, 
these items should be viewed as back-up items and added 
when applying the CUBEST to other Chinese cities. For 
example, slope of sidewalks and bike lanes should be 
added when assessing the built environment in cites with 
hilly terrain, like Qingdao and Chongqing. 

Discussion 

In this study, we developed the instrument designed spe- 
cifically for measuring the urban built environment 
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Table 2 Inter-rater and intra-rater reliability of the CUBEST 



Item 




Inter-rater Reliability (n 


= 309) 






Intra-rater Reliability (n 


= 208) 






Agreement Kappa/ 
(%) ICCs 


PABAK Prevalence 
Index 


Bias 
Index 


Agreement Kappa/ 
(%) ICCs 


PABAK Prevalence 
Index 


Bias 
Index 


Residential Density 




0.853 










0.563 








Row houses of 1—3 stories 


100 


NA 








100 


NA 








Apartment or condos 1—6 stories 


94 


0.776 


0.871 


0.65 


U.UZ 


83 


0.367 


0.663 


0.69 


U.Uo 


Apartment or condos 7—12 stories 


97 


0.821 


0.935 


0.80 


n 03 


88 


0.441 


0.769 


0.77 


n 07 

U.U/ 


Apartments or condos more than 13 
stories 


95 


0.811 


0.909 


0.72 


0.01 


89 


0.500 


0.788 


0.76 


0.01 


Street Connectivity 




0.902 










0.721 








Number of intersections 




0.961 










0.798 








Presence of bus or other transit stops 


99 


0.992 


0.990 


0.38 


0.01 


96 


0.891 


0.913 


0.45 


0.04 


Number of transit stops 




0.972 










0.786 








Accessibility 




0.912 










0.758 








Fast food restaurant 


91 


0.819 


0.819 


0.03 


0.04 


86 


0.689 


0.692 


0.10 


0.01 


Other restaurant 


88 


0.718 


0.767 


0.42 


0.01 


81 


0.522 


0.625 


0.47 


0.04 


Convenience or sma grocery store 


91 


0.764 


0.812 


0.45 


0.01 


78 


0.454 


0.558 


0.44 


0.08 


Supermarket 


96 


0.857 


0.922 


0.68 


U.Uz 


91 


0.691 


0.827 


0.65 


U.U 


Shopping mall or Department store 


93 


0.733 


0.851 


0.67 


0.04 


83 


0.414 


0.663 


0.66 


0.09 


Coffee shop or Tea house or Bar 


94 


0.809 


0.877 


0.60 


n m 
u.u 


86 


0.528 


0.711 


0.63 


n rm 

U.UJ 


Other retail stores 


86 


0.705 


0.728 


0.28 




67 


0.313 


0.346 


0.36 


U.zo 


Other pub lie or government service 
destinations 


90 


0.806 


0.806 


0.01 


0.01 


72 


0.446 


0.442 


0.03 


0.09 


Other recreational destinations 


95 


0.830 


0.909 


0.68 


0.03 


79 


0.313 


0.576 


0.63 


0.12 


Sidewalks/Bike lane 


96 


0.909 


0.922 


0.38 


0.01 


90 


0.792 


0.808 


0.28 


0.03 


Indoor recreation or exercise facility 


99 


0.887 


0.994 


0.97 


0.01 


99 


0.564 


0.971 


0.97 


0.01 


Outdoor sports/playing field 


100 


1.000 








99 


NA 








Public recreational equipments 


99 


0.916 


0.974 


0.83 


U.U I 


96 


0.744 


0.923 


0.84 


u 


Schoo with recreationa faci ities 


100 


NA 








100 


NA 








Neighborhood public park 


99 


0.964 


0.994 


0.91 


U.U I 


96 


0.316 


0.923 


0.94 


U.Uz 


Municipal public park 


99 


0.897 


0.987 


0.94 


0 


1 00 


1 .000 








Public open space that is not a park 


99 


0.91 8 


0.981 


0.87 


0.01 


93 


0.426 


0.865 


0.88 


0.01 


Lake or Beijing-Hangzhou Grand 
Lanai 


97 


0.852 


0.948 


0.81 


0.01 


94 


0.617 


0.885 


0.84 


0.03 


Parking lot or parking garage 


86 


0.712 


0.715 


0.11 


0.05 


73 


0.459 


0.461 


0.14 


0.1 3 


Abandoned building or vacant lot 


98 


0.776 


0.961 


0.91 


U.U 


91 


-0.044 


0.817 


0.91 


U.Uz 


Railroad, bridge, tunnel, or highway 


97 


0.852 


0.948 


0.81 




92 


0.537 


0.846 


0.82 


U.U I 


Walkability (segments with 
sidewalks) 




0.788 










0.360 








Presence of sidewalks 


98 


0.959 








90 


0.894 








Sidewalk width 


94 


0.740 








78 


0.256 








Evenness and sidewalks maintenance 


87 


NA 








83 


NA 








Slope 


99 


NA 








90 


NA 








Obstructions on sidewalks 


75 


0.834 








66 


0.357 








Signs of "Avoid Pedestrians"/ 


83 


0.825 








55 


0.484 









"Slow Down" 
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Table 2 Inter-rater and intra-rater reliability of the CUBEST (Continued) 



Bikeability (segments with bike 
lanes) 




0.766 








0.587 






Presence of bike lane or marked 
shoulder 


99 


0.985 






96 


0.948 






Evenness and Bike lanes maintenance 


96 


0.308 






93 


0.910 






Slope 


99 


NA 






99 


NA 






Presence of speed limit signs on 
motor vehicle anes 


96 


0.925 0.929 


0.21 


0.01 


86 


0.693 0.71 1 


0.25 


0.05 


Street design to reduce volume or 
speed 


84 


0.882 






67 


0.744 






Crossing aids for pedestrians and 
bicyclists to cross the street safely 


86 


0.917 






65 


0.793 






Cars running or parked on bike lanes 


74 


0.757 






58 


0.468 






Cyc ing or walking against the flow of 
traffic on bike lanes 


64 


0.638 






55 


0.464 






Aesthetic 




0.771 








0.710 






Cultural landscape in the 
neighborhood 


84 


0.846 






70 


0.722 






Natural sights in the neighborhood 


63 


0.822 






65 


0.764 






Whole or broken liquor bottles or 
cans 


100 


NA 






99 


NA 






Cigar butts or discarded cigarette 


76 


0.391 






64 


0.314 






Garbage, litter, or broken glass 


61 


0.548 






57 


0.472 






Graffiti on the buildings, signs or walls 


100 


1.000 






97 


NA 






Unattended pets or lost animals 


99 


NA 






96 


NA 







related to physical activity in the Chinese context and con- 
ducted necessary reliability and validity tests. In general, 
most of the items demonstrated at least substantial level 
of inter-rater reliability (Kappa > 0.6, ICCs > 0.75), as well 
as fair to acceptable levels of agreement for the intra-rater 
reliability. Of the 6 dimensions, the accessibility dimension 



showed the highest reliability, which was consistent with 
Brownson's study [13]. 

A low ICC or Kappa statistic could be attributed to ac- 
tual low reliability, a change of the environment between 
the first and the second audit, the subjectivity of judg- 
ments, and/or little variation across segments [28]. Items 



Table 3 Summary of the inter-rater reliability of the CUBEST (N = 309) 



Dimension 
(Number of 
items) 

Residentia 
Density (4) 

Street 

Connectivity 
(3) 

Accessibility 

(21) 

Walkability (6) 

Bikeability (8) 

Aesthetics (7) 

Total Number 
(49) 



ICCs for 
Dimensions 



0.853 
0.902 

0.912 

0.788 
0.766 
0.771 



Kappa Statistic 

NA a Slight Fair Moderate Substantial 
(0-0.2) (0.2-0.4) (0.4-0.6) (0.6-0.8) 

1 1 



Almost Perfect 
(0.8-1.0) 



14 



ICCs 

NA b Poor Fair Good 
(0-0.4) (0.4-0.75) (0.75-1 .0) 



2 

1 1 

2 1 
5 2 



3 
12 



a 2 items were not suitable for computing Kappa statistic due to zero variance or data distribution characteristic. 
b 5 items were not suitable for computing ICCs due to zero variance or data distribution characteristic. 
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Table 4 Summary of the intra-rater reliability of the CUBEST (N = 208) 



Dimension 


ICCs for 








Kappa Statistic 








ICCs 




(Number of 
items) 


Dimensions 


NA a 


Slight 
(0-0.2) 


Fair 
(0.2-0.4) 


Moderate 
(0.4-0.6) 


Substantial 
(0.6-0.8) 


Almost Perfect 
(0.8-1.0) 


NA b 


Poor 
(0-0.4) 


Fair 
(0.4-0.75) 


Good 
(0.75-1 .0) 


Residentia 
Density (4) 


0.563 


1 




1 


2 














Street 

Connectivity 
(3) 


0.721 












1 








2 


Accessibility 
(21) 


0.758 


2 


1 


3 


9 


5 


1 










Walkability (6) 


0.360 














2 


2 


1 


I 


Bikeability (8) 


0.587 












1 


1 




3 


3 


Aesthetics (7) 


0.710 














3 


1 


2 


I 


Total Number 
(49) 




3 


1 


4 


11 


5 


3 


6 


3 


6 


/ 



a 3 items were not suitable for computing Kappa statistic due to zero variance or data distribution characteristic. 
b 6 items were not suitable for computing ICCs due to zero variance or data distribution characteristic. 



Table 5 Construct Validity of the CUBEST 3 : rotated component matrix using factor analysis 



Residential Density b 
Number of Intersections 
Presence of bus or other transit stops 
Commercial destinations' 3 
Physical activity destinations 6 
Parking lot or parking garage 
Railroad, bridge, tunnel, or highway 
Presence of sidewalks 
Sidewalk width 

Evenness and sidewalks maintenance 
Obstructions on sidewalks 
Signs of "Avoid PedestriansV'Slow Down" 
Presence of bike lanes 

Presence of speed limit signs on motor vehicle lanes 

Street design characteristics to reduce volume or 
speed 

Crossing aids for pedestrians and bicyclists to cross 
the street safely 

Cars running or parked on bike lanes 

Cycling or walking against the flow of traffic on bike 
lanes 

Cultural features in the neighborhood 
Natural sights in the neighborhood 
Cigar butts or discarded cigarette 
Garbage, litter, or broken glass 
Graffiti on the buildings, signs or walls 



Residential 
Density 

0.893 



Street 
Connectivity 



0.741 
0.475 
0.583 



0.558 
0.602 



0.418 



Accessibility 



Sidewalk 
quality 



0.703 
-0.417 
0.565 



0.558 
0.556 



0.576 
0.791 



-0492 



Bike lane 
quality 



0.542 



0.911 
-0.509 
0.566 

0.588 

-0.832 
-0.841 



Aesthetics 



1733 
1.765 
1.455 



a After suppressing small coefficients (absolute value below 0.40). 
b Using total score or summary score to represent original items. 
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Table 6 Dimensions and items included in the final CUBEST specific to Hangzhou City 



Dimensions 



Residential 
density 


Street connectivity 


Accessibility 


Sidewalk quality 


Bike lane quality 


Aesthetic 


Types of 


Number of 




Commercial destinations 


Presence of sidewalks 


Presence of bike lanes (Q30) 


Cultural features 


|— |ni nn a 

I I \J kj j M I \J 


intpr^prfinn^ 

II ILCTIjCT^LILJI 1 j \\^-J f 




(025) 




(037) 
















Presence of bus or 


Physical activity destinations 


Sidewalk width (Q26) 


Evenness and maintenance (03 1) 


Natural sights (Q38) 




other transit stops 


(015-022) 










(Q6) 












Number of transit 


Other types of destinations ^ 


Evenness and 


Presence of speed limit signs on 


Cigar butts or 




stops (Q7) 


(Q23-Q24) 


maintenance (Q27) 


motor vehicle lanes (Q32) 


discarded cigarette 












(Q39) 








Obstructions (Q28) 


Street design characteristics to 


Garbage, litter, or 










reduce volume or speed (Q33) 


broken glass (Q40) 








Signs of "Avoid 


Crossing aids for pedestrians and 


Graffiti on the 








PedestriansV'Slow 


bicyclists to cross the street safely 


buildings, signs or 








Down" (Q29) 


(Q34) 


walls (Q4 1) 










Cars running or parked on bike 












lanes (Q35) 












Cycling or walking against the 












flow of traffic on bike lanes (Q36) 





a Row houses; apartment of 1-6, 7-12, more than 13 stories. 

b Fast food; other restaurant; convenience/grocery store; coffee shop or tea house; other retail stores; other service destinations; other recreational destinations. 
c Sidewalks / Bike lane; indoor fitness venues; outdoor sports field; school with facilities; neighborhood park; municipal park; public open space; lake or canal. 
d Parking lot; Railroad, bridge, tunnel, or highway. 



measuring access to destinations changed little over time, 
which ensured higher intra-rater reliability. By contrast, 
some items measuring aesthetic, sidewalk, and bike lane 
quality varied over time (e.g. "Obstructions on bike 
lanes", or "Cigar butts or discarded cigarette"), which re- 
sulted in lower intra-rater reliability. To minimize the 
variation in street environment over time, a two-week 
time interval between the first and second audit was 
chosen, which also providing sufficient washout period to 
avoid memory effect. In addition, necessary subjective 
judgment for a few items can lead to lower inter-rater re- 
liability despite of proper auditor training [9,22]. These 
items included obstacles on sidewalks and bike lanes and 
aesthetic items. 

Little variance or certain data distribution can also 
cause problems (e.g. lower Kappa statistic due to high 
prevalence effect). The PABAK coefficient was reported 
alongside of the original Kappa coefficient, which was es- 
pecially necessary when the prevalence index or the bias 
index was high. The item "Presence of community park" 
presented perfect observer agreement (96%), however, 
the original Kappa coefficient was only 0.316 due to high 
prevalence index. The PABAK adjusted for prevalence ef- 
fect reached 0.923, much higher than unadjusted value. 
Adjustments for prevalence and bias effect increased the 
Kappa coefficient in 4 items in the inter-rater reliability 
test and 15 items in the intra-rater reliability test. 
Hoehler [29] is critical of the use of the PABAK because 
the effects of bias and prevalence on the magnitude of 



kappa are themselves informative and should not be ad- 
justed for. Alongside the obtained values of Kappa and 
the PABAK, we also reported the prevalence index and 
bias index to provide more valuable information as rec- 
ommended by other authors [27,30]. 

Factor analysis was used to test construct validity. The 
six extracted principal components were basically in ac- 
cordance with the theoretical framework of the CUBEST. 
A few items showed larger factor loading on dimensions 
to which they did not belong, however, these results can 
be explained. For example, the commercial buildings 
usually were located at places where streets are well- 
connected to increase business visibility. The analysis 
correspondingly indicated that the commercial destin- 
ation items loaded highly on the factor of street connect- 
ivity instead of accessibility. Similarly, "Evenness and 
sidewalks maintenance" could not only be used to meas- 
ure the sidewalk quality, but also be in reflection of the 
aesthetic quality. Another example is that, the "Presence 
of bus or other transit stops" loaded strongly on bike lane 
quality instead of street connectivity because the public 
bicycle rental stations are usually built near bus stops to 
facilitate the traffic movement. 

A 91-item Environment in Asia Scan Tool - Hong 
Kong (EAST HK) was developed by Cerin et al. to ob- 
jectively assess aspects of the neighborhood environment 
hypothesized to affect walking in Hong Kong and similar 
ultra-dense Asian metropolises [9]. There were differ- 
ences in several aspects between the CUBEST and 
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EAST HK. First, items hypothesized to affect cycling be- 
havior were added in the CUBEST given that cycling 
remains one of the most popular form of transport in 
China. Second, the EAST HK included 41 items as a list 
of destinations. However, we had special focus of the 
CUBEST on two most important types of destinations, 
that was commercial (8 items) and physical activity des- 
tinations (8 items), and had other types of destinations 
simplified to keep the questionnaire as short as possible. 
A simplified instrument would be a more feasible option 
for large-scale survey. Third, items measuring safety 
from crime were not included in the CUBEST due to 
consideration of hard judgment for raters during a short 
stop in the street. In addition, additional items were 
added, like the presence and number of transit stops to 
better capture variation of street connectivity. 

We acknowledge the limitations of this study. One is 
the lack of criterion validity test for the instrument. GIS 
data could be used as criterion measure to test the valid- 
ity of residential density, street connectivity, and land- 
use mix. Unfortunately, poor accessibility and high cost 
make the use of GIS data impossible in this and many 
other studies conducted in China [31]. The other limita- 
tion is that this study took place in one city of China, 
which may limit the application of the CUBEST to other 
Chinese cities. However, we evaluated the CUBEST in 
three different types of administrative planning units, 
which to some extent ensured enough variations in the 
environmental features. When applying the CUBEST in 
other Chinese cities, we suggest that some items should 
be added or modified to capture specific built environ- 
ment features to different cities. 

Conclusions 

The CUBEST is a reliable and valid instrument that can 
be used to assess the physical activity-related built envir- 
onment in Hangzhou, and potentially other cities in 
China (with suitable modifications). 
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